Automatic clustering of people featured in images within an image collection based on face information has been studied and built into commercial photo organization systems. However, existing clustering techniques can perform poorly when there are large pose variations or occlusions, which are not uncommon in consumer images. In general, the precision rate of the grouping of faces in clusters can be relatively high. That is to say, for most of the clusters, each cluster can contain the faces of a single person, with major clusters usually corresponding to major family members for example. However, the recall rates of face clusters can be unsatisfactory with only a fraction of an individual's images in the photo collection included in his/her corresponding face cluster. This can be exacerbated by the provision of a number of small/singleton clusters of, for example, major family members that cannot be easily allocated to the major clusters. When the number of non-major clusters is large, this could essentially result in the requirement for a user to label faces one by one in a collection. The recall rate of face clusters can be reduced by two things: firstly, missed detections in a face detection stage; and secondly, ill-illumination, image blurring and pose variations, as well as occlusion of human faces due to an uncontrolled capture environment.
Clothing information has been employed in a number of prior approaches in order to provide additional cues to face information for the purpose of human identification. In prior methods, the clothing information and face information are fused at the distance metric level. That is, these methods normalize and then sum up (with or without weights) the separate distance matrices computed based on different information into a single distance matrix, which is then used to cluster images of the person. This information fusion method is expensive in terms of required computation and memory. Also, valuable discriminative information is likely to be lost in the matrix processes.